How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice based on Over 60 Replicated Studies

Instrumental variable (IV) strategies are commonly used in political science to establish causal relationships, yet the identifying assumptions required by an IV design demanding and it remains challenging for researchers evaluate their plausibility. We replicate 61 papers published three top journals from past decade (2011-2020) document several troubling patterns: (1) often miscalculate first-stage F statistics, overestimating strength of IVs; (2) most rely on classical asymptotic standard errors, which severely underestimate uncertainties around two-stage-least-squared (2SLS) estimates; (3) majority replicated studies, 2SLS estimates much bigger than ordinary-least-squared estimates, ratio is negatively correlated with IVs studies where not experimentally generated, suggesting potential violations exclusion restriction; such a relationship weaker generated IVs. To improve practice, we provide checklist avoid these pitfalls recommend zero-first-stage test local-to-zero procedure guard against failure assumptions.